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Background: The Danish National Registry of Patients (DNRP) is a potentially valuable 
resource for monitoring national trends in the use of chemotherapy and evaluating the benefits 
and harms of alternative treatments among colorectal cancer (CRC) patients in Denmark. 
However, the validity of chemotherapy reporting in the DNRP is unknown. In this study, we 
evaluated the validity of the DNRP for identifying the receipt of chemotherapy and specific 
treatments, and the timing and number of treatments among CRC patients, using medical records 
and pharmacy data as the reference standard. 

Methods: We selected a random sample of CRC patients with lymph node involvement who 
were diagnosed at Aarhus University Hospital (n = 25) or Aalborg University Hospital (n = 25) 
from 2009 to 2010. Administration dates, specific treatments, and number of treatment courses 
were abstracted for the 1 80 days post diagnosis from the DNRP, medical records, and pharmacy 
production databases. The prevalence of chemotherapy, timing of first administration, and 
number of courses were described. DNRP data were compared with the reference standard for 
each hospital, and the kappa, sensitivity, specificity, positive and negative predictive values, 
and 95% confidence intervals were calculated for the receipt of any chemotherapy and specific 
treatments. 

Results: The prevalence of chemotherapy was 72% and 68% among CRC patients treated in 
Aarhus and Aalborg, respectively, with >90% of patients without distant metastasis receiving 
treatment within 90 days from diagnosis. Patients received on average 4.6 and 4.7 treatment 
courses in Aarhus and Aalborg, respectively. Kappa, sensitivity, and specificity of chemotherapy 
reporting in the DNRP was high (>0.88), but the sensitivity of individual chemotherapies 
varied by hospital. 

Conclusion: The validity of chemotherapy reporting in the DNRP was high, although some 
variation by hospital exists. The DNRP represents a population-based nationwide resource 
that can be used to provide timely and accurate evaluations of chemotherapy use among CRC 
patients in Denmark. 

Keywords: chemotherapy, colorectal cancer, administrative data 

Introduction 

In 20 11 , colorectal cancer (CRC) was the third most commonly diagnosed cancer among 
both men and women in Denmark.* Treatment for CRC may include a combination of 
surgical resection, radiation therapy, and/or chemotherapy. When a CRC has spread to 
the lymph nodes, chemotherapy is typically indicated.^ Over the past ten years, random- 
ized controlled trials have documented the efficacy of a number of new chemotherapeutic 
regimens for the treatment of CRC that have led to substantial improvements in survival 
and quality of life. ^ ^ Population-based medical registries, such as those developed in 
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Denmark, may be a useful resource for monitoring the use 
and dissemination of new therapies into chnical practice, and 
provide a unique setting to evaluate their benefits and harms 
among cancer patients who are frequently underrepresented 
in randomized controlled trials including older adults and 
patients with multiple comorbidities.' 

Until 2004, the Danish Cancer Registry routinely recorded 
whether patients diagnosed with cancer received various 
cancer-directed treatments (including chemotherapy) within 
4 months from the date of diagnosis. However, detailed 
information regarding the use of specific regimens and dates 
of administration was not available and limited the utility of 
the data for research purposes. After 2004, the Danish Cancer 
Registry completely ceased to collect information regard- 
ing chemotherapy. As a result, alternative data sources for 
monitoring and evaluating the use of chemotherapy in the 
Danish population are indicated. The aim of this study was 
to evaluate the validity of the Danish National Registry of 
Patients (DNRP), a hospital-based medical registry, to identify 
the receipt of chemotherapy and specific treatments, as well 
as the timing and number of treatment courses among CRC 
patients using information from patient medical records and 
pharmacy production data as the reference standard. 

Materials and methods 

Data sources and validation sample 

The Danish national health care system provides the entire Dan- 
ish population with unrestricted access to tax-supported public 
health services.* Through the use of a unique ten-digit civil 
registration number, medical registry and administrative data 
can be obtained and linked.' Our study relied upon linkage of 
the Danish Cancer Registry, the DNRP, patient medical records, 
and two hospital pharmacy production databases located within 
Aarhus University Hospital and Aalborg University Hospital. 
These two large hospitals were selected because they provided a 
sufficient number of new CRC cases each year, and we obtained 
approval to access patient medical records and pharmacy pro- 
duction data from the regions where these hospitals are located 
for the conduct of validation studies. 

We identified all patients with an incident CRC diagnosed 
between January 1, 2009 and December 31, 2010 from the 
Danish Cancer Registry and included those who had lymph 
node involvement using the tumor, node, metastasis (TNM) 
staging system (ie, N > N 1 We further restricted our sample 
to individuals who also had a diagnosis of CRC in the DNRP 
within 14 days of the diagnosis date reported by the Danish 
Cancer Registry, because this registry does not identify the 
hospital of diagnosis.' Using this algorithm, we identified all 



patients who were diagnosed at Aarhus University Hospital 
(2009-2010) and Aalborg University Hospital (2010 avail- 
able only). Due to resource constraints associated with the 
review of patient information, we selected a random sample 
of 25 patients from each hospital (Figure 1) to include in our 
analysis. Because cancer care is centralized at the regional 
level in Denmark, it is reasonable to assume that if a patient 
was diagnosed with cancer at a hospital in Aarhus or Aalborg, 
then that patient would receive their chemotherapy at the same 
hospital. Aarhus is a university town and as a result the back- 
ground population is relatively young compared with Aalborg. 
Aarhus University Hospital performs surveillance examinations 
for patients with inflammatory bowel disease and inherited 
conditions, such as Lynch syndrome, while diagnostic work-up 
of symptomatic patients is performed at other hospitals in the 
region. As such, the patient population diagnosed with CRC 
at Aarhus University Hospital tends to be slightly younger 
than the population diagnosed at Aalborg University Hospital. 
Surgery and chemotherapy for CRC were also provided out- 
side of university centers in the county hospitals in Viborg, 
Heming, Horsens (colectomy only), and Hjorring during the 
study period; however, patients diagnosed with CRC at these 
hospitals were not included in the present study. 

Reporting of chemotherapy 
from DNRP, medical records, 
and pharmacy production system 

For each CRC patient, we obtained chemotherapy informa- 
tion for the 1 80 days following the diagnosis date as reported 



All colorectal cancer patients diagnosed in Denmark (2009-2010) 
n=8,251 



Lymph node involvement^ 
n=3,104 



Identified hospital where the colorectal cancer was diagnosed'' 
n=3,030 



Patients diagnosed in Aarhus or Aalborg from 2009-2010 
n=333 



Patients diagnosed in Aarhus 
(2009-2010) 
n=114 



Patients diagnosed In 
Aalborg (2010) 
n=101 



Random sample of 
25 patients 



Random sample of 
25 patients 



Figure I Study flow chart of patients selected for Inclusion In the validation study. 
Notes: ''Regional lymph node Involvement was identified using the tumor, node, 
metastasis (TNM) staging system (N ^ Nl); ''using an administrative algorithm 
requiring a hospital diagnosis of colorectal cancer (International Classification of 
Diseases (ICD)-IO codes: CI 8-20) + 14 days from the diagnosis date, as reported 
by the Danish Cancer Registry. 
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by the Danish Cancer Registry (the "study period") using the 
DNRP, the patient medical record, and the hospital pharmacy 
production system. We selected this study period because 
most patients should initiate their first course of adjuvant 
or postoperative chemotherapy during this time period (if 
indicated), and it would be unlikely to observe treatment 
for recurrence or progression during this window. We 
were particularly interested in initial treatments containing: 
5-fluorouracil (5-FU) or capecitabine only, oxaliplatin in 
addition to 5-FU (FOLFOX) or capecitabine (XELOX) with- 
out bevacizumab, and bevacizumab in addition to FOLFOX, 
XELOX, irinotecan + 5-FU (FOLFIRI), or 5-FU. These three 
groups are mutually exclusive. 

In Denmark, oncology care is delivered in hospitals, and 
treatments and procedures are coded in the DNRP using 
Danish treatment codes. We retrieved all patient records with 
treatment codes indicating chemotherapy administration or the 
use of specific agents and regimens for the treatment of CRC 
during the study period (Supplementary Table 1). Electronic 
patient medical records were reviewed by a physician (TD), 
and chemotherapy data including the receipt of any chemo- 
therapy and specific treatments, the dates of first administra- 
tion of each treatment, and the number of treatment courses 
received during the study period were abstracted. Finally, we 
obtained data from the Aarhus University Hospital Pharmacy 
and Aalborg University Hospital Pharmacy production sys- 
tems, which record orders placed by oncologists at the hospital 
for production in the hospital pharmacy. Data regarding the 
specific agents and regimens and dates of production were 
obtained for each patient during the study period. 

Comparison of chemotherapy reporting 
in DNRP, medical records, and pharmacy 
production system 

During the majority of the study period, the pharmacy pro- 
duction systems did not record oral chemotherapy agents, 
because they were disbursed to the hospital in bulk orders 
and not on a per-patient basis. Because the patient medical 
record captures oral chemotherapy use, we combined these 
two data sources to serve as the reference standard for all of 
the validation analyses. For the description of the number 
of treatment courses administered during the study period, 
we relied solely on the pharmacy production data (excluding 
those treated with oral chemotherapy alone), because report- 
ing of production dates was most straightforward. 

We reported measures of validity for chemotherapy 
reporting in the DNRP compared with the reference 
standard. Specifically, we estimated the sensitivity, 



specificity, positive and negative predictive values, and 
their corresponding 95% confidence intervals. We also 
reported overall agreement comparing the two data sources 
using Cohen's kappa and corresponding 95% confidence 
intervals. We evaluated reporting of the receipt of any che- 
motherapy and specific treatments. The time from cancer 
diagnosis to treatment was described for each data source 
among patients without distant metastasis in the sample 
(<30 days, 31-60 days, 61-90 days, and 91+ days). In 
addition, the number of months with any reported treatment 
was used as a proxy for the number of treatment courses 
completed,'" and we described this distribution within the 
study sample using the pharmacy production data and the 
DNRP (excluding those treated with oral chemotherapy 
only). All analyses were performed using SAS version 
9.2 (SAS Institute Inc, Cary, NC, USA). This study was 
approved by the Danish Data Protection Agency (2006-53- 
1346 and 2006-53-1396). 

Results 

We identified 8,251 individuals with an incident CRC 
diagnosed between January 1, 2009 and December 31, 
2010 in Denmark. Among the 3,104 patients who had 
regional lymph node involvement, 3,030 could be matched 
to a hospital where they were diagnosed. There were 1 14 
patients diagnosed at Aarhus University Hospital between 
2009 and 2010, and 101 patients diagnosed at Aalborg 
University Hospital in 2010. Our study sample of 50 
patients was randomly selected from this group (Figure 1). 
The median age at diagnosis was slightly higher for patients 
diagnosed in Aalborg compared with those diagnosed in 
Aarhus (70 years versus 68 years) and lower for those 
receiving chemotherapy compared with those who did not 
(Table 1). There were more rectal cancers included in the 
Aalborg sample compared with the Aarhus sample (56% 
versus 40%). The majority of CRCs in the overall sample 
had advanced stage (T3-T4, 90%), had four or more lymph 
nodes involved (N2, 62%), but did not have distant metas- 
tases (MO, 66%). According to the reference standard, the 
prevalence of chemotherapy was 72% and 68% in Aarhus 
and Aalborg, respectively. 

The comparisons of any chemotherapy and specific treat- 
ments reported by the reference standard and the DNRP for 
each hospital are shown in Table 2. Overall, the measures 
of validity and agreement for identifying the receipt of any 
chemotherapy were high. For Aarhus University Hospital, 
the sensitivity, specificity, positive predictive value, negative 
predictive value, and kappa were 94%, 100%, 100%, 87%, 
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Table I Characteristics of 50 randomly selected patients diagnosed with colorectal cancer in Denmark by hospital, 2009-2010 
Characteristic Aarhus University Hospital Aalborg University Hospital 



Chemotherapy received Chemotherapy received 

Total sample (reference standard) Total sample (reference standard) 
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Notes: ^Includes administration of 5-FU or capecitabine alone; ''includes administration of oxaliplatin in combination with 5-FU (FOLFOX) or capecitabine (XELOX), but not 
with bevacizumab; ^includes administration of bevacizumab in combination with FOLFOX, XELOX, irinotecan, and 5-FU (FOLFIRI), or 5-FU; ^based on pharmacy production 
data only, excluding patients reporting treatment with capecitabine only in the DNRP. 

Abbreviations: Col %, column percentage; DNRP, Danish National Registry of Patients; Row %, row percentage; 5-FU, 5-fluorouracil; NOS, not otherwise specified; 
SD, standard deviation. 



and 90%, respectively, and for Aalborg University Hospital 
were 100%, 88%, 94%, 100%, and 90%, respectively. 

The measures of validity and agreement for specific 
treatment reporting varied by treatment and were lower 
for patients diagnosed at Aarhus University Hospital 
compared with those diagnosed at Aalborg University 
Hospital (Table 2). For patients diagnosed at Aarhus, the 
validity of bevacizumab reporting in the DNRP was high, 
with a sensitivity, specificity, positive predictive value, 
and negative predictive value of 75%, 100%, 100%, and 
95%), respectively. Reporting of all specific treatments 
from the DNRP among patients diagnosed at Aalborg 
resulted in perfect sensitivity of 100%, specificity >94%, 
and kappa ^91%. 

Consistent with current treatment guidelines,^ adjuvant 
chemotherapy was initiated within 90 days from the diag- 
nosis date in >90% of the non-metastatic CRC patients 
in both hospitals, regardless of the data source used for 



reporting (Figure 2). After excluding patients receiving 
oral chemotherapies exclusively, 73% and 43% of colon 
cancer patients and 33% and 80% of rectal cancer patients 
treated in Aarhus and Aalborg, respectively, completed five 
or more courses of treatment according to the pharmacy 
production data. Overall, the DNRP reported a similar dis- 
tribution of the number of treatment courses administered 
by CRC site as compared with the pharmacy production 
data (Figure 3). 

Discussion 

We found that the DNRP accurately recorded the receipt of 
chemotherapy administered within 1 80 days from diagnosis 
of colorectal cancer, as well as the date of first chemo- 
therapy administration and number of treatment courses 
when compared with the treatment information obtained 
from the patient medical record and pharmacy production 
systems. However, the ability of the DNRP to record the 
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use of specific chemotlierapy treatments varied by hospital, 
where the sensitivity of individual chemotherapy agent 
reporting was higher among patients treated in Aalborg 
compared with Aarhus. 

This is the first study to assess the validity of the DNRP 
for identifying chemotherapy treatment in CRC patients and 
as such provides a framework for future studies seeking to 
determine the accuracy of treatment coding for other cancer 
sites. Prior studies in the United States have shown similar 
results when examining the validity of administrative data, 
such as Medicare claims, to identify the receipt of chemother- 
apy among older CRC patients." '^ These data have become 
widely used to describe trends in the utilization and compara- 
tive effectiveness and safety of a variety of treatments over 
time in the United States, but are limited to patients diagnosed 
with cancer at age 65 years and older.""" 

A major strength of the current study is our ability to 
identify and link population-based patient information across 
multiple data sources by using the Danish CPR number. As 
a result, we were able to obtain data from complementary 
sources including patient medical records and pharmacy 
production data to construct a robust reference standard. 
The DNRP is not restricted to specific patient populations 
(eg, the elderly), as is often the case in the United States, 
and thus can provide more generalizable inferences regard- 
ing chemotherapy use and effectiveness for CRC patients 
of all ages. 

Our study has some limitations. We were only able to 
validate chemotherapy administered in two large hospitals in 
Denmark for the time period from 2009 to 2010, which may 
impact the generalizability of our findings to other hospitals 
and time periods (ie, 2004-2008). However, oncology care 
in Denmark is considered to be relatively uniform due to 
the nationalized structure of health care and the National 
Cancer Plan I and II,'* " and as such, treatment protocols 
and standards should be similar across hospital settings. 
We observed that recording of particular chemotherapy 
treatments varied according to hospital, and as such analy- 
ses seeking to identify the use of these treatments will be 
limited to specific hospitals. In addition, our analysis did not 
directly examine the reliability of chemotherapy reporting 
in the DNRP. However, analysis of the number of treatment 
courses indirectly provides evidence that the DNRP reliably 
captures chemotherapy treatment over the study period, given 
that the distribution in the two data sources is similar. Finally, 
we relied upon data regarding nodal involvement from the 
Danish Cancer Registry, which is known to be incomplete.^" 
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0-30 days 31-60 days 61-90 days 91+ days 0-30 days 31-60 days 61-90 days 91+ days 

Aarhus University hHospital Aalborg University Hospital 

■ DNRP Medical record ■ Pharmacy data , Dl^^p Medical record ■ Pharmacy system 

Figure 2 Proportion of patients with non-metastatic disease according to time between diagnosis and first administration of chemotherapy, by data source and hospital. 
Aarhus University Hospital is shown in the left panel, while Aalborg University Hospital is shown in the right panel. Date of diagnosis was defined according to the Danish 
Cancer Registry. 

Abbreviation: DNRP, Danish National Registry of Patients. 



However, this is unlikely to impact the generalizability of our 
findings because treatment reporting in the DNRP is unlikely 
to be related to the completeness of TNM reporting in the 
Danish Cancer Registry. 

In conclusion, we assessed the validity of the DNRP for 
identifying the receipt of any chemotherapy and specific 
treatments. Generally, the DNRP accurately identified the 



receipt of any chemotherapy. However, the sensitivity of 
specific treatment reporting depended upon individual hos- 
pital coding practices. The DNRP, in combination with data 
from the Danish Cancer Registry and other medical registries 
and clinical quality databases, provides investigators with a 
unique opportunity to conduct high quality health services 
and cancer outcomes research. 
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Figure 3 Proportion of patients completing a range of chemotherapy courses within 180 days from cancer diagnosis by hospital and cancer site, excluding patients treated 
with oral chemotherapy only. The number of completed courses is estimated as number of months with at least one record of administration of chemotherapy. Treatment 
data from the DNRP are shown in black and the pharmacy production data are shown in gray. 
Abbreviation: DNRP, Danish National Registry of Patients. 



submit your manuscript | www.dovepress.com 

Dovepress 



Clinical Epidemiology 20 1 3:5 



Dove^iress 



Chemotherapy in Danish colorectal cancer patients 



Acknowledgments 

We would like to thank Lis Estrup and Anni Christensen 
from Aarhus University Hospital Pharmacy and Gitte 
Sondergaard Nielsen from Aalborg University Hospital 
Pharmacy for their efforts in obtaining the pharmacy pro- 
duction system data. 

Disclosure 

The study was supported by a grant from the Danish Cancer 
Society (R73-A4284-13-S17) and from the Karen Elise 
Jensen Foundation. 



References 

1. Statens Serum Institut. Cancerregisteret 2011, tal og analyse. [Cancer 
Registry 2011, Figures and Analysis]. Available from: http://www.ssi. 
dk/Sundhedsdataogit/Registre/~/media/lndhold/DK%20-%20dansk/ 
Sundhedsdata%20og%20it/NSF/Registre/Cancerregisteret/Cancerreg- 
isteret%20201 l.ashx. Accessed April 22, 2013. Danish. 

2. The Danish Colorectal Cancer Group. The Danish Colorectal Can- 
cer Group Guidelines 2012. Available from: http://www.dccg.dk/ 
retningslinjer/indeks.html. Danish. Accessed April 22, 2013. 

3. Andre T, Boni C, Mounedji-Boudiaf L, et al. Oxaliplatin, fluorouracil, 
and leucovorin as adjuvant treatment for colon cancer. N Engl J Med. 
2004;350:2343-2351. 

4. Hurwitz H, Fehrenbacher L, Novotny W, et al. Bevacizumab plus iri- 
notecan, fluorouracil, and leucovorin for metastatic colorectal cancer. 
NEnglJMed. 2004;350:2335-2342. 

5. Murthy VH, Krumholz HM, Gross CR Participation in cancer 
clinical trials: race-, sex-, and age-based disparities. JAMA. 2004;291: 
2720-2726. 

6. Ministry of Health and Prevention. Health Care in Denmark, 
Copenhagen, Denmark, 2008. Available from: http://www.sum.dk/ 
Aktuelt/Publikationer/-/media/Filer%20-%20Publikationer_i_pdf/2008/ 
UK_Healthcare_in_dk/pdf.ashx. Danish. Accessed July 24, 2013. 

7. Frank L. Epidemiology. When an entire country is a cohort. Science. 
2000;287:2398-2399. 



8 . Greene FL, Page DL, Fleming ID, et al. AJCC Cancer Staging Manual. 
6th ed. New York, NY: Springer; 2002. 

9. Helqvist L, Erichsen R, Gammelager H, et al. Quality of lCD-10 
colorectal cancer diagnosis codes in the Danish National Registry of 
Patients. Eur J Cancer Care (Engl). 2012;21:722-727. 

10. Dobie SA, Baldwin LM, Dominitz JA, et al. Completion of therapy 
by Medicare patients with stage III colon cancer. J Natl Cancer Inst. 
2006;98:610-619. 

1 1 . Lund JL, Sturmer T, Harlan LC, et al. Identifying specific chemothera- 
peutic agents in medicare data: a validation study. Med Care. 2013;51: 
e27-e34. 

12. Warren JL, Harlan LC, Fahey A, et al. Utilify of the SEER-Medicare 
data to identify chemotherapy use. Med Care. 2002;40:IV55-IV61. 

13. LundJL, Sturmer T, SanoffHK, BrookhartA, Sandler RS, Warren JL. 
Determinants of adjuvant oxaliplatin receipt among older stage II and III 
colorectal cancer patients. Cancer. 2013;119:2038-2047. 

14. Mack CD, Glynn RJ, Brookhart MA, et al. Calendar time-specific 
propensity scores and comparative effectiveness research for stage III 
colon cancer chemotherapy. Phannacoepidemiol Drug Saf. 2013;22: 
810-818. 

15. SanoffHK, Carpenter WR, Freburger J, et al. Comparison of adverse 
events during 5-fluorouracil versus 5-fluorouracil/oxaliplatin adjuvant 
chemotherapy for stage III colon cancer: a population-based analysis. 
Cancer. 2012;118:4309-4320. 

16. SanoffHK, Carpenter WR, Martin CF, et al. Comparative effectiveness 
of oxaliplatin vs non-oxaliplatin-containing adjuvant chemotherapy for 
stage III colon cancer. J Natl Cancer Inst. 2012;104:21 1-227. 

17. SanoffHK, Carpenter WR, Sturmer T, et al. Effect of adjuvant chemo- 
therapy on survival of patients with stage III colon cancer diagnosed 
after age 75 years. J Clin Oncol. 2012;30:2624-2634. 

1 8 . National Cancer Plan, Copenhagen, Denmark: National Board of Health; 
2004. Available from: http://www.sst.dk/publ/Publ2004/The_Nat_ 
Cancer_Plan_02.pdf Accessed July 24, 2013. 

19. National Board of Health. National Cancer Plan II, Copenhagen, 
Denmark: National Board of Health, 2005. Copenhagen, Denmark: 
National Board of Health; 2004. Available from: http://www.sst.dk/publ/ 
Publ2005/PLAN/kraeftplan2/KraeftplanII_UK/KraeftplanII„UK_med. 
pdf Accessed July 24, 2013. 

20. Ostenfeld EB, Froslev T, Friis S, et al. Completeness of colon and 
rectal cancer staging in the Danish Cancer Registry, 2004-2009. Clin 
Epidemiol. 2012;4 Suppl 2:33-38. 



Clinical Epidemiology 2013:5 



submit your manuscript | www.dovepi ess.com 

Dovepress 



333 



Lund et al 



Doveore^s 



Supplementary table 



Table SI Treatment codes used to identify chemotherapy for 
the treatment of colorectal cancer in the DNRP 



Type of chemotherapy 



DNRP treatment code 



Any chemotherapy for colorectal cancer BWHA I , BWHA2, BOHJ 1 9B 



S-FU or capecitabine 

Oxaliplatin alone 

Oxaliplatin + S-FU (FOLFOX) 

Oxaliplatin + capecitabine (XELOX) 

Irinotecan alone 

Irinotecan + S-FU (FOLFIRI) 

Bevacizumab alone 



BWHA! 10, BWHA 1 23 

BWHA 1 08 

BWHA23 1 

BWHA222 

BWHA2I2 

BWHA2I2 

BOHJI9B 



Abbreviations: 5-FU, 5-fluorouracil; DNRP, Danish National Registry of Patients. 
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